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ATTACHMENT OF ADDITIONAL INFORMATION TO STORED OR 
STREAIffiD IMAGE 

5 The present invention relates to a method for storing ot streaming images, wherein the 
images can be further processed after the storing or during the streaming. 

The invention further relates to a system for capturing and storing and/or streaming 
images, wherein the images can be further processed after the storing or during the 
10 streaming, comprising 

- a detector adapted to capture data; 

- a video processor connected to the detector, the video processor being adapted to_^ 

process the captured data; 

- a CPU connected to the video processor, the CPU being adapted to provide the 
15 video processor with parameters for the processing; 

- a file processor and/or a stream processor connected to the video processor and the 
CPU. 

v.. 

BACKGROUND OF THE INVENTION 

20 

It is for example of great interest to be able to process images taken by infrared cam- 
eras (IR-cameras). IR-cameras are used for capturing images showing the temperature 
variance for different objects in a view. Such thermal images are for example used 
when searching for defects in devices since defect parts of . devices are often heated be- 
25 fore they break. 

An IR-camera typically captures information of temperatures from about - 60 degrees 
Celsius to about 2000 degrees Celsius. This range is called the thermal camera dy- 
namic range. Typically a thermal image will show the background ambient temperature 
30 and some hot or cold objects that will be subject for inspection. Background ambient 
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will vaiy from outdoor winter temperature to a hot summer day temperature. The in- 
spected object (a thermal anomaly) differ from the background as a hot or cold object 
within or outside the typical background range. Thus a thermal image will comprise a 
much smaller range than the thermal camera dynamic range. An image could be pre- 
5 pared from only this interesting temperature span and then be delivered-to for example 
a customer. This is today a common way to handle temperature images. Thus the im- 
ages taken by infrared cameras are processed by the image provider and then trans- 
ferred to customers as a view where only a part of the initially captured data is shown. 
A problem with this is that this view not can be processed by the customers any fur- 
10 ther. The customer may have a different opinion than the provider of the image about 
what is the interesting part of the view. Furthermore, if ihe temperature span for visu- 
alising the background is chosen to be for example 20-40 degrees Celsius, extreme 
eyents, such as a hotspot (a thermal anomaly like an overheated electric conductor) or 
an explosion, will saturate in the image thermal range and not be shown in the image. 

15 

Another approach today is to provide all the initially captured data to the customer. A 
problem with this method is that the customer not is able to see the image at all before 
he has processed it and to process the initially captured data the customer needs to have 
4pdicated software. 

20 

SUMMARY 

An object of the invention is to improve the storing and/or streaming and distribution 
of images to customers whwein the images can be further processed by the ciitstomer 
25 2^^er the storing or during the streaming. 

A further object of the present invention is to provide a method and a system for stor- 
ing and distributing these images such that the user is able to view the image without 
needing any dedicated software and such that the user at the same time has access to 
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the initially captured data such tiiat he can choose which data that is interesting for him 
and process the image by using dedicated software. 

These objects are achieved in a method as initially described where the method com- 
5 prises the steps of: ^ ^ - . 

- choosing a span of the initially captured data, this span being adapted to be showi| 
as an image to a user; 

- storing or streaming this chosen span of the data as an image in a file format or 
onto a stream to which additional information can be attached and which file for- 

10 mat or steam could be processed by at least one standard application; 

- storing or streaming all or a part of the initially captured data as additional infor- 
mation to this file format or stream; ^ 

- storing or streaming parameters needed for the processing of the image as further 
additional information to tiie file or stream. 

15 

TTie objects are also achieved in a system as initially described wherein the file proces- 
sor and the stream processor comprises: 

- receiving means adapted to receive data from the video processor and instruction^ 
and parameters from the CPU; 

20 - first storing means and first streaming means respectively, both connected to the 
receiving means and adapted to store and stream respectively one portion of the 
data received from the video processor as an image in a file format and onto a 
stream respectively to which additional information can be attached and which file 
format and stream can be processed by at least one standard application, the storing 

25 and streaming of one portion of the received data being performed according to the 
instructions received from the CPU; 

- second storing means and second streaming means respectively, both connected to 
the receiving means and adapted to store and stream all or a part of the data re- 
ceived from the video processor as additional information to the file and the stream 
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respectively, the storing and streaming being performed according to the received 
instructions or according to pre-set definitions; 
- third storing means and third streaming means respectively, both connected to the 
receiving means and adapted to store and stream respectively parameters needed for 
5 the processing of the image as further additional information to the file and the 

stream respectively, the parameters being provided to the third storing and stream- 
ing means through the receiving means from the CPU. 

Hereby a method and a system are provided by which a receiver of the data is able to 
10 bbth view an image without using any special software and to post-process the image 
according to his own desires. The receiver has access to all or at least to the interesting 
part of the initially captured data. The receiver also has access to all the parameters 
necessary for the processing of the image. 

15 The initially data could be captured by a camera. The camera could be an infrared 
6bnera and the image is thus an image containing information about temperatures. 

Preferably the used file format is chosen among the following formats JPEG, PNG, 
TIFF, MJPEG and DV. 

20 

. Suitably the storing or streaming of all or a part of the initially captured data as addi- 
tional information to the file format or stream is performed in a 16 bits format Hereby 
a good resolution is obtained. 

.25 In one embodiment of the invention only the initially captured data not being chosen 
for the image is stored or streamed as additional information to the file format or 
stream. Hereby memory is saved since no data needs to be stored or streamed twice. 

' i 

Suitably the initially captured data is sub-sampled before storing or streaming it as ad- 
30 ditional information to tiie file format or stream. Hereby memory is saved. 
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Preferably an initial image is captured over a span of about - 60 to 2000 degrees Cel- 
sius. 

• • ■ ' 1^, ■ 

5 Suitably the storing or streaming of parameters as additional information comprises 
storing or streaming one or more of the following: the calibration constants for the 
camera, the compensation for the background, the conditions for the detector, maximal 
and minimum temperatures, information about the colour scales and measurement and 
set-up parameters. Hereby the post-processing of the image is possible. 
0 ■ 
Advantageously the users are provided with the software needed for the processing of 
the additional data. 



In one embodiment of the invention the method comprises capturing images following 
15 each other, making up a movie and storing these images in a row of file formats as de- 
scribed above. ^ 

Preferably the stream processor is adapted to perform compression according to DV or 
MJPEG. 

20 

Suitably the file processor is adapted to store the file comprising the image, the addi- 
tional data and the parameters in a non volatile or a volatile memory. v 

BRIEF DESCRIPTION OF THE DRAWINGS 

25 

Fig. 1 shows a histogram of an ordinary image captured by an infi:ared camera. 
Fig. 2 shows a file format to which additional information can be attached, ^ 
30 Fig. 3 is a schematic view of a system according to the invention. 
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DETAILED DESCRIPTION OF EMBODIMENTS 

Fig. 1 shows a histogram of an ordinary image captured by an injfrared camera (IR 
5 camera). The total temperature span foi^a common IR-camera ranges, from about - 60 
tq 2000 degrees Celsius. As can be seen in Fig. 1 only a small span of the temperature 
interval is interesting. This is the normal case. For example in this case, the marked 
span between 20 and 90 degrees Celsius would be the most interesting part. Thus a 
common way to present these images is to cut out the interesting span, compress and 

1 0 store this span of Ihe image in some kind of file format. However, if tiie span between 
20 and 90 degrees Celsius is chosen, the possibility to see extreme events, such as an 
plosion, is lost. On the other hand tiie human eye would not resolute a viewable im- 
age if the whole span of the camera between -60 and 2000 degrees Celsius would be 
used for the image. Furthermore different spans could be interesting for different pur- 

1 5 poses. Thus a possibility to post-process the image would be of interest for many users. 

As described above another approach is to provide all the initially captured data to the 
^sers. However, special software is then needed to be able to see and process the im- 
age. 

20 

According to the present invention an image that is most interesting according to the 
provider of the image is compressed and stored according to common standards in a 
file format to which additional information could be attached. Alternatively titie image 
i§ streamed onto a stream to which additional information can be attached. This ^'most 
25 interesting" image can then be presented by standard applications and all users can see 
this image without any special software and without processing the image. 

Fig. 2 shows a file format 1 1 to which additional information can be attached. In the 
embodiment of the invention shown in Fig. 2 the file format is JPEG (Joint Photo- 
30 .graphic Expert Group). PNG (Portable Network Graphics) or TIFF (Tagged Image File 
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Format) are examples of other possible formats. Other similar formats are of course 
also possible. In the embodiment shown in Fig, 2 the "most interesting" image is stored 
in a first part of the JPEG file called APPO 13 . In this embodiment the image is stored 
according to JFIF (JPEG File Interchangeable Format). Accordmg to the invention 
5 some or all of the initially captured data are store^d in a tag to the file, in this case called 
APPl 15. APP stands for application and indicates that different applications can place 
information in different tags to the file. The name of the information provider is indi- 
cated in the beginning of each tag. If a destructive compression such as JPEG is used 
all of the initially captured data is preferably attached to the image in the APPl tag 15. 

10 The reason is that the JFBP image is slightly destroyed in the compression and wont be 
able to use for processing. If on the other hand a none destructive or lossless compres- 
sion such as PNG is used it would be enough to attach the rest of the initially captured 
data in a tag to the PNG file. Also other types of discriminations to the initially cap- 
tured data are possible. For example different temperature spans could be of interest 

15 for different users and different purposes. 

Furthermore some radiometry parameters has to be attached. This is in the embodiment 
shown in Fig. 2 accomplished by adding additional sub-tags to the APPl main tag 15. 
In the JPEG format up to 15 main tags could be provided. Fxirthermore each tag can be 

20 provided with additional sub-tags. In this embodiment the radiometry parameters are 
placed in a sub-tag to the APPl tag 15. The parameters are needed for the processing 
of the image and comprise for example measurement and set-up parameters. It could 
be the settings of the camera at the time for the capturing of data, the conditions for the 
detector, the calibration constants for the camera, colour scale information, measuring 

25 points and maximal and minimum temperatures. Further examples of additional infor- 
mation that can be added to the image are textual comments, for example about what 
can be seen on the image, other kinds of textual information and soxmd information, 
for example comments firom the photographer. Furtheraiore the distance fi-om the cam- 
era to the measurement object, the positions of objects in the image, such as co- 

30 ordinates from a GPS, and information about the customer and the customer's applica- 
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tions could be added to the file. In the embodiment shown in Fig. 2 both the initially 
captured data and all the parameters are attached in the APPl tag 15. All necessary pa- 
rajneters firom the camera and the object(s) could of course be attached to the image. 

5 When the user wants to see the. thermal image he opens the JPEG file. The JFIF image 
is shown to the usct since this image can be shown by standard applications. No spe- 
cial software is needed to show this image. To be able to process the image however, 
the user needs to use the additional data attached to the JPEG file in the APPl tag 1 5. 
To process this data special software is needed. This software could be provided firom 

10 the camera provider to the user. Alternatively such software could be provided as 
scripts or applets on a homepage. The user can then install the software fi-om the 
homepage. The software could be a new application that allows processing of the im- 
age data or it could be a software enabling already existing applications to process the 
image data. During the processing of the image the parameters that were attached to 

15 th^ JPEG image in the sub-tag to the APPl tag 15 are used for calculations. 

This method of storing tiie "most interesting" image together with all the initially cap- 
tured data and together with the parameters necessary for calculations and processing 
makes it possible for flie users to process the image and all users have access to all the 
20 initially captured data. Furthermore the user can view the "most interesting" image 
without any special software. 

The initially captured data stored as additional information in the APPl tag 15 is in one 
embodiment stored in a 16-bits format. 16-bit resolution is needed to resolute a thermal 

25 image in the mKelvin range. Using an 8-bits format is also possible but the resolution 
will of course be worse. Furthermore this additionally stored data could be compressed 
according to known art. The compression should however not be a destructive com- 
pression since the data should be further processed by the user. It is also possible to 
sub-sample, i.e. only store every second pixel of the image, to save memory. Sub- 

30 sampling will of course reduce spatial resolution. However, this reduction can be ac- 

8 
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ceptable as long as the amount of low-pass filtering is less than the thermal spatial 
resolution. 

The method according to the invention could also be applied for moving images. The 
5 images are simply stored, or actually streamed, in files as described above where the \\ 
files are following each other in a stream like the DV (Digital Video) stream format 
and/or file formats made for motion like the non-standard M- JPEG/MJPEG/MJPG. A 
way of adding thermal information to a DV stream, normally used in digital Camcor- 
ders, could be to add the information to the existing sound channel. Intelligent com- 
10 pressing or data reduction needs to be applied according to what have been mentioned 
above. ^ ' 

There are some possible variants within the scope of this invention concerning the 
storing or streaming of the initially captured data. As already mentioned all the initially 
15 captured data does not have to be attached to the file. It is also possible that the pro- 
vider of the image processes the image, for example for compensating for technical 
conditions during the capturing of the image and for technical conditions of the cam- v. 
era, in a first step. Then the user/customer can process the image in a second step ac- 
cording to his own interests. 

20 

Fig. 3 is a schematic view of a system according to the invention. The system is here 
an IR camCTa. It comprises a detector 21 adapted to capture data. The data is in this 
case thermal data about the objects seen by the camera. Furthermore the camera com-V 
prises a video processor 23 connected to the detector 21. The video processor 23 is 

25 adapted to process the captured data. The processing comprises for example filtering of 
the captured data and focusing of the image. The processing could also comprise the 
discrimination of the captured data to achieve an image viewable by the human eye. 
Instructions about what kind of discrimination that is desired is provided to the video 
processor 23 fi:om a CPU 25 connected to the video processor 23. Other parameters ^ . 

30 needed for the processing could also be provided firom the CPU 25. The provider of the 
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data could thus change the instructions and parameters regarding different possibilities 
of processing the captured data through the CPU 25. 

The camera comprises further a file processor 27 and/or a stream processor 29 con- 
" 5 nected to the video processor 23 and the CPU 25 . The file processor 27 and the stream 
processor 29 comprises according to the invention receiving means 3 1 ,3 T adapted to 
receive the processed data from the video processor 23 and instructions and parameters 
from the CPU 25. The processed data received from the video processor 23 could be 
all the initially captured data. Alternatively the data could have been sub-sampled in 
10 Ihe video processor 23 or some parts of the data could have been sorted out Further- 
more it is possible that the discrimination of the initially captured data to a visual im- 
age is done in the video processor 23 . In this case both this image and all or a part of 
the initially captured data is transferred to the file processor and/or to the stream proc- 
essor. The discrimination process to create a visual image is in another embodiment of 
15 the invention performed in the file processor and the stream processor respectively. In- 
stractions about the discrimination are provided to the video processor 23 or tiie file 
processor and the stream processor respectively from the CPU 25. The other mentioned 
instructions from the CPU 25 could be instructions about how to treat and store the re- 
ceived data and the parameters. The parameters are parameters needed for the post- 
20 processing of the thermal data. Examples of such parameters are given in connection to 
Fig. 2. 

The file processor comprise further according to the invention first storing means 33, 
second storing iheans 35 and third storing means 37 all connected to the receiving 

25 means 33. In the same way the stream processor comprise first streaming means 33 
second streaming means 35' and third streaming means 37' all connected to the re- 
ceiving means 31'. The first storing and streaming means 33,33' are adapted to store 
and stream respectively one portion of the data received from the video processor 23 in 
a file format 1 1 (Fig. 2) or onto a stream respectively to which additional information 

30 can be attached and which file format 1 1 and stream can be processed by at least one 
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Standard application. This portion should be stored (or streamed) as an image and in 
one embodiment it is stored in a JPEG file format. As described above other formats 
are also possible. Which portion of the received data that should be stored (or 
streamed) as an image is given by the instructions received j&om the CPU 25. 

The second storing and streaming means 35,35' are adapted to store and stream re- 
spectively all or a part of the received data as discussed above as additional informa- 
tion to the file 11 or the stream, the storing and streaming being performed according^:^ 
to the instructions received from the CPU or according to pre-set definitions. The sec- 
10 ond storing and streaming means 35,35' could thus be provided with fixed instructions 
about how to store and stream the data. Otherwise the instructions are provided from 
theCPU25. 

The third storing and streaming means 37,37' are adapted to store and stream respec-^ 
15 tively parameters needed for the processing of the image as fiarther additional informa- 
tion to the file 1 1 or the stream. The parameters are provided to tiie third storing and 
streaming means 37,37' through the receiving means 31,31' from the CPU 25. Exam- 
ples of the parameters are given in the description of Fig. 2 above. 

20 The stream processor 29 is adapted to perform compression according to for exampld ' 
DV, MJPEG or broadcast or pomt to point videoconferencing systems such as H323. 
The file processor 27 is adapted to store the file 1 1 with the image, the additional data 
and the parameters in a non volatile or a volatile memory. Some examples are 
PCMCIA (Personal Computer Memory Card Intemational Association) and a hard- 

25 drive or a flash file system in the camera. 

The embodhnents described above are all concerning temperature images taken by an 
IR-camera. The invention could however also be apphed for other techniques. Ultra- 
sound analyses and vibration analyses are two examples where data is captured and 



II 



wo 02/091741 



PCT/SE02/00857 



presented to the us^ as an image and thus two examples of techniques where the pres- 
eik invention could be applied. 
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CLAIMS 

1 . A method for storing or streaming images, wherein the images can be further proc- 
essed after the storing or during the streaming, characterised in that it comprises 

5 the steps of: 

- choosing a span of the initially captured data, this span being adapted to be shown 
as an image to a user; 

- storing or streaming this chosen span of the data as an image in a file format or 
onto a stream to which additional information can be attached and which file for- 

10 mat or stream could be processed by at least one standard application; 

- storing or streaming all or a part of the initially captured data as additional infor- 
mation to this file format or stream; 

- storing or streaming parameters needed for the processing of the image as further 
additional information to the file or stream. 

15 . . 

2. A method according to claim 1, comprising capturing the initially data by a camera. 

3. A method according to claim 2, wherein the capturing is performed by an infrared 
camera and the image being an image containing information about temperatures. 

20 

4. A method according to any one of the preceding clainais 4, wherein the used file 
foraiat is chosen among the following formats JPEG, PNG, TIFF, MJPEG and DV. 

5. A method according to any one of the preceding claims, wherein the storing or 
25 streaming of all or a part of the initially captured data as additional information to 

the file format or stream is performed in a 16 bits format. 

6. A method according to any one of the preceding claims, comprising storing or 
streaming only the initially captured data not being chosen for the image as addi- 

30 tional information to the file format or stream. 
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7. A method according to any one of the precedmg claims, comprising sub-sampling 
of the initially captured data before storing or streaming it as additional information 
to the file format or stream. 

5 i - - - . • . 

8. A method according to any one of the preceding claims, comprising capturing an 
initial image over a span of about - 60 to 2000 degrees Celsius. 

9. A method according to any one of the preceding claims, wherein the storing or 
10 streaming of parameters as additional information comprises storing or streaming 

one or more of the following: the calibration constants for the camera, the compen- 
sation for the backgroimd, the conditions for the detector, maximal and minimimi 
temperatures, information about the colour scales and measurement and set-up pa- 
rameters. 

15 

10. A method according to any one of the preceding claims, comprising providing the 
-ft Tisers with the software needed for the processing of the additional data. 

1 1. A method according to any one of the preceding claims, comprising capturing im- 
20 ages following each other, making up a movie and storing these images in a row of 

file formats as described in any one of the claims 1-10. 

12. A system for capturing and storing and/or streaming images, wherein the images 
can be fiirther processed afl:er the storing or during the streaming, comprising 

25 - a detector (21) adapted to capture data; 

- a video processor (23) connected to the detector (21), the video processor (23) be- 
ing adapted to process the captured data; 

- a CPU (25) connected to the video processor (23), the CPU (25) being adapted to 
provide the video processor (23) with parameters for the processing; 
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- a file processor (27) and/or a stream processor (29) connected to the video proces- 
sor (23) and the CPU (25), 

characterised in that the file processor (27) and the stream processor (29) comprised . 

- receiving means (3 1.3 1') adapted to receive data firom the video processor (23) and 
5 instructions and parameters firom the CPU (25); ' 

- .first storing means and first streaming means (33,33') respectively, both connected 

to the receiving means (3 1,3 1') and adapted to store and stream respectively one 
portion of the data received from the video processor (23) as an image in a file 
format (1 1) and onto a stream respectively to which additional information can bei^. 
LO attached and which file format (1 1) and stream can be processed by at least one 

standard application, the storing and streaming of one portion of the received data 
being performed according to the instructions received fi-om the CPU (25); 

- second storing means and second streaming means (35,35') respectively, both con- 
nected to the receiving means (3 1,3 1 ') and adapted to store and stream respectively 

15 aU or a part of the data received fi:om the video processor (23) as additional infor-V 
mation to the file and the stream respectively, the storing and streaming being per- 
formed according to the received instructions or according to pre-set definitions; 

- third storing means and third streaming means (37,37') respectively, botii con- 
nected to the receiving means (3 1 ,3 1 ') and adapted to store and stream respectively 

20 parameters needed for the processing of the image as fiirther additional information 
to the file (1 1) and the stream respectively, the parameters being provided to the v. 
tiiird storing and streaming means (37,37') through tiie receiving means (37,37') 

from the CPU (25). 
25 13.A system according to claim 12, characterised in that it is a camera. 

14. A system according to claim 13, characterised in that it is an infrared camera and ^ ; 
the detector (21) is adapted to c^ture thennal data. 
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15. A system according to any one of the claims 12-14, wherein the storing and 
' streaming means (33,33%35,35 ',37,37') are adapted to store or stream the data and 
parameters in JPEG, PNG, TIFF, MJPEG or DV. 

5 16. A system according to any one of the claims 12-15, wherein the second storing and 
streaming means (35,35') are adapted to store the data in a 16 bits format 

17. A system according to any one of the claims 12-16, wherein the second storing and 
streaming means (35,35') are adapted to store and stream only the data not having 

10 beenstoredby the fiurst storing and streaming means (33,33'). 

18. A system according to any one of the claims 12-17, wherein tiie second storing and 
streaming means (35,35') are adapted to sub-sampling the data before storing and 

• streaming it. 

15 

19. A system according to any one of the claims 12-18, wherein the stream processor 
(29) is adapted to perform compression according to DV or MJPEG, 

20. A system according to any one of the claims 12-19, wherein the file processor (27) 
20 is adapted to store the file comprising the image, the additional data and the pa- 
rameters in a non volatile or a volatile memory. 
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